\begin{answer}
    With a single example, the expectation of the estimator is given by
$$
\mathbb E_{s\sim p(s)\atop a\sim\pi_0(s, a)}\frac{\frac{\pi_1(s, a)}{\pi_0(s, a) }R(s, a)}{\frac{\pi_1(s, a)}{\pi_0(s, a)}} = \mathbb E_{s\sim p(s)\atop a\sim\pi_0(s, a)}R(s, a)
$$
This is clearly biased, unless $\pi_0 = \pi_1$.

\end{answer}
